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ABSTRACT 

Standard theory of differential games focuses the study on two-person 
zero-sum games, and treat N-person games separately and differently. In this 
paper we present a new equivalent formulation of the Nash equilibrium strategy 
for N-person differential games. Our contributions are the following: 

1) Our min-max formulation unifies the study of two-person zero-sum with that 
of the general N-person non zero-sum games. Indeed, it opens a new avenue 
of systematic research for differential games. 

2) We are successful in applying the finite element method to compute 
solutions of linear-quadratic N-person games. We have also established 
numerical error estimates. Our calculations, which are based upon the 
dual formulation , are very efficient. 

3) We are able to establish global existence and uniqueness of solutions of 
the Riccati equation in our form, which is important in synthesis. This, 
to our knowledge, has not been done elsewhere by any other researchers. 

This paper's particular emphasis is on the duality approach , which is 
motivated by computational needs and is done by introducing N + 1 Language 
multipliers : one for each player and one "joint multiplier" for all 

players. For N-person linear quadratic games, we show that under suitable 
conditions the primal min-max problem is equivalent to its dual min-max 
problem , which is actually a saddle point and is then computed by finite 
elements . Numerical examples are presented in the last section. 
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§0. Introduction 
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Consider an N-person differential game with linear dynamics 


( 0 . 1 ) 


x(t) = A(t)x(t) + B 1 (t)u 1 (t) + ... + B^(t)u^(t) + f(t), 0 i t i 1, 

x(0) = x Q € R n , 


where u, € U, s L = 
i i in^ 

of the i-th player P^; 

2 2 

functions, f € L n s L 
variable. 


2 i 

L (0,T;3l ) is the control variable under the command 

A, B^ are proper n x n, n * m^ matrix valued 
(0,T;R n ) is the inhomogeneous term and x is the state 


N 

An N-tuple of controls u = (u^,...,u^) € U = n IL is called an open- 

loop strategy. Associated with each player P^ is a cost functional J^(x,u) 

(1 2 i 2 N) incurred in a game due to a strategy u and the outcome x of 
(0.1) that is generated by u. The case when J\ is quadratic of the form (3.1) 
in §3 will be of particular interest to us. 

Each player P^ wishes to minimize his cost J^. We say that u = (u^.u^,..., 
u^) forms an (optimal) equilibrium strategy if 


( 0 . 2 ) 


J i (x,u 1 , . 


■V - j ± ( x >v 


••* u i-l ,v i* u i+l* 


■V- 


ISiSH, 


for all v i £ 1^. Such a strategy allows all players to play individual optimal 
strategies simultaneously. Therefore the questions of its existence, uniqueness, 
solutions and computations constitute the most important study in the theory of 
N-person differential games. 
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Standard theory of differential games (e.g. [6], [3]) focus the study on 
two-person zero-sum games, and treat N-person games separately and differently. 
For two person zero-sum games, the concept of an equilibrium strategy coincides 
with that of a saddle point. One then proceeds to use either the Pontryagin 
(Friedman, Issacs) minimaximum principle or the Bellman dynamic programmxng to 
derive necessary conditions for equilibrium. To compute optimal strategies, one 
must either solve (usually) a two-point boundary value problem of ODEs or a PDE 
(the Bellman-Hamilton-Jacobi equation). 

For N-person games, the pioneering work was done by Lukes and Russell [9]* 
Their basic point of view, which was inherited in most of the subsequent papers 
on this subject, actually was to regard N-person differential games as a more 
complex N-simultaneous optimization problem. From (0.2), they regard as the 
optimal control for the i-th player when other players are using respective 
strategies u^ , . . . , . . .u^. So they proceeded to use the primal, dual, 

or feedback synthesis methods to solve 


r 


Min J i (x,u 1 , . . . ,u i _ 1 ,v i ,u i+1 , .. . ,u N ) 


subject to 


x = Ax + Z B.u. + B.v. + f 

jl»i J J 11 


V, 


x(0) = x,. 


for players i = 1,2,...,N. So u^ can be obtained by differentiating with 
respect to v^, while holding other players' individual optimal strategies fixed. 
This yields N simultaneous equations for u^,...,u^. The solvability of these 
equations gives N necessary conditions in general. Even if these N equations 



J 


can be solved simultaneously, it is not certain (except perhaps in the linear- 
quadratic case, wherein the invertibility of certain operators is a sufficient 
condition) that the derived controls u^,...,u^ indeed form an equilibrium 
strategy, since vL, . . . ,u^ mutually interfere through the system dynamics. As 
a matter of fact, the game-theoretic nature of the problem seems to be lost in 
this approach. 

In this paper, we present a new approach to N-person games - we show that 
an N-person game can also be formulated into a min-max point problem (§1). 

This formulation gives a necessary and sufficient condition for the existence 
of equilibrium strategies. This min-max problem is primal . Later on, we will 
see that under certain conditions this min-max problem is actually a saddle 
point problem. In this sense, we see that our work has unified the theory 
two-person zero-sum games with the theory of N-person non zero-sum games. 

In §2, we formulate the dual of the primal problem, which becomes a max-min 
problem. In the dual formulation, system dynamical equations like (0.1) are 
eliminated, thus the new max-min problems is unconstrained . The dual problem is 
formulated in terms of N + l Lagrange multipliers p,(0 S i S N): one multiplier 

p^ for each player (1 S i S N) and one "joint multiplier" Pq for all 
players. 

Beginning from §3, we specialize to the quadratic cost case. We formally 
synthesize the closed-loop equilibrium strategy and derive the (new) Riccati 
equation (3.13) which is different from those in other formulations (see e.g. 

[6], [9]). 

§4 deals with the variational formulation of the dual problem. Here we 
make several assumptions which ensure the tractability of the dual problem. Then 
the "primal-dual equivalance theorem" is established. The important existence 
and uniqueness of equilibrium strategy is proved in Theorem 4.7. 
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In §5, we establish the global existence and uniqueness of the solution of 
the Riccati equation. 

§6 studies finite element approximations. Our work here is motivated by 
similar work on the Ritz-Trefftz and the finite element methods for optimal 
controls (see, e.g. [2], [10]). To our knowledge, this is the first time the 
finite element method is applied to differential games. 

Numerical results are given in the last 3 7. 

In our sequel. Part II [3], we will again use the basic formulation in §1, 
but combine it with the penalty and the finite element methods, and compare our 
numerical results from these different approaches. 



5 


§1. Equilibrium Strategy as Min-Max Point 

We first formulate a sufficient condition which states that an equilibrium 
strategy can be found as a min-max point. In a two-person zero-sum 
game, such a saddle point formulation is given a priori. However, for an 
N-person game our formulation seems to be completely new; it forms the basis 
for all of our future discussions. 

For each u € U, one can solve x from (0.1) and determine J^(x,u)(l 2 i 5 N) . 
Thus each J^(x,u) is a functional on (u^,...,u^), so we define 

(1.1) ^^(u^> • • • » u^) = J^(x, u^ , . . . , u^) . 


For u , v 6 U, u = (n^,...,!^), v = (v^...,^), let 


N 


(1.2) F(u,v) = 2 [£ ± (u) - ^(v 1 )], v 1 a (u^ . . . .u^.v^u^ , . . . , 1 ^) . 


Lemma 1.1 If u = (u^,...,!^) satisfies 


(1.3) sup F(u ,v) 2 0, 
v«J 


then u is an equilibrium strategy. Conversely, if u is an equilibrium 
strategy, then (1.3) holds. 

^ ^ ^ ^ 

Proof : Assume that (1.3) holds. Choose v = (u^, . . . ,u^_^,v^,u^ + ^, . . . , 1 ^) , 

where v^ € is arbitrary. Then 

F(u ,v ) 2 sup F(u , v) 2 0. 
v<EU 


(1.4) 
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But 


^ ft ft ft 

F(u , v ) = ^ i (u 1 , . . . , 1 ^) - « i (u 1> , 


,U i-l’ V i ,U i+l’ 


7* 

-V 


which is less than or equal to 0 by (1.4). So (0.2) is satisfied; u is an 
equilibrium strategy. 

* 

Conversely, if u is an equilibrium strategy, then 


(1.5) ^ ± (u 1 , . . . ,u N ) - ^(i^,. 




i-l ,v i ,u i+l** 


,u N ) < 0 , Vv ± € U.. 


Summing (1.5) from 1 through N, we get F(u ,v) 2 0 , Vv £ U. Hence (1.3) 
holds. □ 


Theorem 1.2 If 


( 1 . 6 ) 


inf sup F(u,v) < 0 
u€U v«J 


or 


( 1 . 6 ') 


min sup F(u,v) 2 0 
u€U v£U 


is satisfied, then the differential game has at least one equilibrium strategy. 


Proof: Under (1.6), we have at least one u 6 U such that sup F(u,v) 2 0 
Vv € U. By Lemma 1.1, u is an equilibrium strategy. Same conclusion holds 
for (1.6'). □ 
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Remark 1.3 In the above proof, we see that if we choose v = u, then 

0 = F(u,v) S sup F(u,v) - 0, 
v 

therefore sup F(u,v) =0. We see that it is impossible to have sup F(u,v) < 0. 
v v 

Thus (1.6) is ruled out. An equilibrium strategy exists if and only if 

(1.6") min sup F(u,v) = 0. 

u€U vdU 

A simple corollary is that if (u,v) solves 

(1.7) F(u,v) = min max F(u,v) = 0, 

u<EU v€U 

then u is an equilibrium strategy. □ 

Remark 1.4 In the discussion above, nowhere have we used the linear 

dynamics of (0.1). Therefore Theroem 1.2 and Remark 1.3 are valid under 
the general setting of [6]. 

Therefore, the question of finding an equilibrium strategy is reduced to 
solving the min-max problem (1.7) or (1.6"). 

From now on, we signify the Sobolev space 

k , . 

H k = H k (0,T) a {y : [0,T] ]R n | ||y|| . = 2 ||(Tr) J yll 2 < “>• 

n n h k dt 

n n 


We define 
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where 


( 1 . 8 ) 


If the 
we can 


(1.9) 


where 


( 1 . 10 ) 


( 1 . 11 ) 


and 


1 N 

J(x,u;X,v) = J(x, u^ > • • • , ; x ,...,x ,v^,...,v^) 


N 


= 2 [J ± (x jU^,...,U^) — J ± (x , , . . . ,U^_^ > • . • > U^) ] : 


1 N 1 N i 

X = (x ,...,x ) € [H n ] and each x is the solution of 


1 


x 1 = Ax^ + B^U- + ... + B, ,u, + B.v. + + ... + B„u T + f 

1 1 i-1 i-l l i l+l i+I N N 


x 1 (0) = x r 


given differential game has at least one equilibrium strategy, then 
consider solving 


min max J(x,u;X,v), 

x,u X,v 

(DE)-O [DE]=0 


M l 

(DE) = x - Ax - 2 B.u. - f, x € H 

J-l J J 

N 2 1 
[DEI = 2 | (DE)| , X(0) = (x (0),.. 

i-l 


subject to x(0) = Xq, 


N 

x N (0)) = 


(*, 


O’ 


,x Q ) = 


X, 


0» 


(DE) = 


.i 

x 


-Ax - 


N 

2 

j-l 

j*i 


B.u. 

j 1 


Vi 


- f , x^" € , subject to x^(0) 


= x„ 


( 1 . 12 ) 



Suppose that the cost functional is given as 


J ± (x,u) =/ q h^(t,x(t) ,u^(t) , . . . ,u^(t) )dt + g i (x(T)) 


In our framework, we can define the Hamiltonian as 


N i 

(1.13) H(t,x,u,X,v,q Q ,q) = __Z [^(t.xCt) ,0^(0 , . . . ,1^(0) - h^t.x (t) , 


^(t) , . . . ,u i _ 1 (t) ,v i (t) > u i+1 (t) , • • • .^(t)) 1 


N 


+ <q (t) ,A(t)x(t) + 2 B.(t)u. (t) + f(t)> 

U 1=1 1 


N 


N 


+ 2 <q (t) ,A(t)x (t) + I B.(t)u (t) +B.(t)v (t) +f(t)>, 

j=l J J 
3 * i 


i=l 


where q = (q^q^t • • • ,q N ) • The Pontryagin minimaximum principle can be stated 

as follows: Assume that (u,v) is a rain-max point for min max F(u,v) subject to 

u v 


(DE) = 0, [DE] = 0; let x,X,qg,q satisfy the canonical equations 


(1.14) 


dx(t) 

dt 


r— H(t,x,u,X,v,q n ,q) 

3q o ° 


; x(0) = 


x„ 


q o =q o 


(1.15) 


dt 


gT“ H(t,x,u,X,q Q ,q) 
q i 


; S £ (0) = x Q , 1 Si S N ; 


q=q 


( 1 . 16 ) 


dq Q (t) 

dt 


— H(t,x,u,X,v,q Q ,q) 


S q 0 CT) 


x=x 


N 3 
i=l dx 


x : 


:=x(T) 
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(1.17) 


dq ± (t) 

dt 


3x 


1 H(t,x,u,X,v,q Q ,q) 


q ± (T) = - 


x=x 


a , i. 

-8,(x ) 

dx 


i“i 


x =x (T) 


Then, we have, necessarily , the Hamiltonian at a min-max point for all time 
t € (0,T) : 


(1.18) H(t,x,u,X,v,q n ,q) = min max H(t,x,u,X,v,q ,q) 

(x,u) (X,v) 


Alternatively, we can also use the dynamic programming approach. Define 
"the value of the game" V(t,^, 2) by 


N 


(1.19) V(t.?,S) = 


T i 

min max 2 {/ [h ± (t ,x(t) .u^t) , . . . , 1^(0 )-h i (t,x (t) ,u 1 (t) , 

u v i=l T 

* . . , t ) , v ± ( t ) , u^^ (£),•••, (t ) ) ] dt 

+ g.(x(T)) - g ± (x i (T)) } 


subject to 


N 

x(t) = A(t)x(t) + 2 B (t)u (t) + f(t), x(t) = § € H n 

i=l 1 


N . 

x^t) = A(t)x 1 (t) + 2 B. (t)u. (t) +B (t)v.(t) +f(t) , x X (t) = K X € R n , 

j = 1 J J X x 

j^i 1 S i S N, 


on [t,T] 3 t, with 2 = € [R n ] N . 


If (1.19) is well-defined, under suitable assumptions (cf. [6J, §4), we have 


the Issacs equation 



1L 


(1.20) r^VCr^S) + mifl 


u€ n jl 

i=i 


N 

max {<7„ V(t,5,H) ,A(t)5 + 2 B . (t)u. + f (t)> 

N m, t, ' 


v € n R 
i=l 


1=1 


i x 


R 


N 


N 


+ 2 <V V(t,5,S), A(t) 5 + 2 B <T)u. + B (t)v + f(x)> 
1-1 5 j-1 J 2 -p n 

j*i * 


N . 

+ 2 [h ,(T,£,U) - h (T.c’-.v 1 )] = 0 , 

1=1 1 


(v — (u^ , . . . , u^_^ > » • • • » u,,) ) 


with the terminal conditon 


N 


(1.21) V(T,5,S) = 2 [g AO - g t (5 )]• 

i=l 1 


This leads further to the Bellman-Hamilton-Jacobi equation 


a N 

(1.22) V(t,x,X) + <7 x V(t,x,X), A(t)x + 2 B^t^t ,x,X, V,V X V) + f(t): 

i_l 


N 


N 


+ 2 <V .V(t,x,X),A(t)x + 2 B.(t)u (t,x,X,V V,V V) + 

i=l x j-1 J J xx 

l* 1 


+ B.(t)v i (t,x,X,7 x V,7 x V) +f(t): 


N 


i rL, 


+ 2 [h. (t,x,u(t,x,X, V V, V V)) - h . (t ,x ,v (t,x,X,V V.O)] = 0, 

x. a. i X A 


where u(t,x>,X,qg,q) and v(t,x,X,qQ,q) are "feedback controls" for u and v 


satisfying 
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min m max m H(t,x,u,X,v,q Q ,q) = min m max^ H(t,x,u,X,v,q Q ,q) 
uQIR 1 vSTR 1 vSIR 1 ufflR 1 


= H(t,x,u (t,x,X,q Q ,q) , X,v(t,x,X,q 0 ,q) ,q Q ,q) 

for t 6 [0,T], x €E n , X € [R n f, q = (q^...^) 6 [R n ] N , q Q 6 R n . 

Comparing (1.21), (1.22) with [6] p. 293, (8.2.5), (8.2.6), for example, we 
see that our B-H-J equation is a single equation (in contrast with a system of N 
equations), but of 1 + n(N + 1) independent variables (in contrast with 1 + n 
variables) . 

Throughout the above paragraphs, that the min-max point (x,u;X,v) corresponds 
to an equilibrium strategy depends on whether the value of (1.9) is 0 or not. 

This important issue will be addressed in our future papers. For linear quadratic 
games, a good answer can be found in (4.27) of Theorem 4.7. 


§2. Duality Theory 

We consider the following inf-sup problem 


(P) 


inf sup {J (x,u;X,v) | J as in (1.8), (x,u) € H 1 x U subject to (DE) = 0, 
x, u X,v n 

(X,v) € [H^] N x u subject to [DEj =0 as in 

( 1 . 10 ), ( 1 . 11 ) > . 


This constitutes the primal problem . Associated with (P) is the dual problem 


(D) 


sup inf L (p n ,p) , 
2 2 N u 

p 0 a n p6IL n ] 


where p = (p 1 ,...,p N ) and 
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l (p 0 .p) = LCPq^Pjl. • • • >P n ) = inf sup L(p Q ,p;x,u;X,v) 

x,u X,v 


with the Lagrangian Li x [L^] N x x u x [H^] N x u defined by 

N 

(2.1) L(p ,p;x,u;X,v) s J(x,u;X,v) + <p ,x - Ax - 2 B.u. - f> 

j-1 2 2 L 

J n 

N .i i N 

+ 2 <p.,x - Ax - 2 B.u. - B.v. - f> 

1-1 1 11 2 

j?i n 


for x,X satisfying x(0) = x^ , X(0) = X^ = (Xq,...,Xq). 

From now on we say that (x,u) or (X,v) is feasible if (x,u) € x u 

n 

satisfies (1.10) and (X,v) € [H^] N x u satisfies (1.12). Similarly, (Pg>P) 
is feasible if (p Q ,p) € a [L^] N . 


We are now in a position to state the fundamental theorem in this paper. 


Theorem 2.1 (Duality Theorem) Assume that J(x,u;X,v) is convex in (x,u) and 
concave in (X,v), for all (x,u) and (X,v) satisfying differential constraints, 


continuous m 


in H 1 x U x [h L ] N x U and 
n 1 n J 


(AO) inf sup J(x,u;X,v) s c < 09 . 

(x,u) (X,v) 

feasible feasible 


Then there exists (pg,p) which is a max-min point for (D) with L(pg,p) - c. 
Furthermore, if (x,u;X,v) is a min-max point for (P), then 


L(p n >p) = max min 

2 , rT 2,N 
p 0 a „ pftL n ] 


( 2 . 2 ) 


L (P 0 »P) 
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= max 

p n €L 2 

^0 n 


min 


min 

(x,u) 


, rT 2,N 

P ^ [ 1* J / n \ _ 

n x(0)=x 0 


max 

(X,v) 

X(0)=X Q 


L(p 0 ,p;x,u;X,v) 


min 

(x,u) 

feasible 


max J(x,u:X,v) 
(X,v) 
feasible 


= J(x,u;X,v) . 


We proceed to prove the theorem. 

For any given (x,u) € x U, let 


(2.3) y( x ’ u ) 5 SU P J(x,u;X,v), 

(X, v) 
feasible 


and also define 


N 


(2.4) cf>(x,u,p) = sup {J(x,u;X,v) + 2 <p i> (DE) i >|X € [H*] N ,v e u » x <°) = x n > 

X,v i=l 

P ~ Cp-L* - . . , P N ) f [L^] }, 


By (AO), we know that there exists at least one feasible (x,u) such that 


(2.5) sup J(x,u;X,v) = \|f(x,u) < + 00 . 
(X,v) 
feasible 


From now on we need only study t(x,u) and ^(x,u,p) for those (x,u) satisfying 


(2.5). 
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Lemma 2.2 (Weak Duality) For any (x,u) satisfying (2.5), the functional 
$ (x, u, p) defined above is convex in p and 


( 2 . 6 ) 


inf 

CfT 2 1 N 

p«i L n i 


<*>(x,u,p) > t(x,u) 


holds. 


Proof: Simple verification. 


Lemma 2.4 (Strong Duality) Assume that J(x,u;X,v) is concave in (X,v) for 
all (X,v) € [H^ ] N x U, X(0) = X Q . Then for any (x,u) € H* x U, x(0) = x Q , 


we have 


(2.7) inf <f>(x,u,p) =\Kx,u). 

, rT 2 i N 
p6[L n ] 


Proof : If \(r(x,u) = + 00 , then (2.7) holds trivially by Lemma 2.2. So we assume 
that (2.5) holds. The arguments in [7, p. 846-847] immediately apply. We define 
two convex sets 


Y = { (a,0) € B. x [L^] N I a > ^(x,u)} 


2,N 


Z a {(a,b) € R x [Lj 


a 2 J(x,u;X,v), b = (b lt ...,b ), 

b. = x 1 - Ax 1 - B v. - Z B.u. - f, 
i i i . , . i J 

j/i J J 


x 


'(0) = x n , i = 1, . . . ,N. ) 
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Then it is easily checked that Y fl (interior of Z) = 4 > since when b = 0 6 [LT] , 


a < J(x,u;X,v) 2 sup J(x,u;X,v) 

(X,v) 

feasible 


for any (a,0) € [interior of Z 1 which is obviously nonempty. So by the 

separation theorem (see, e.g. [11], p. 38, Theorem 3.3.3), Y and Z can be 

2 N 

separated weakly in R x [L fl ] : 


( 2 . 8 ) 


a + 2 <q ,b > < r • a V(a 1 ,b) 6 Z, (a_,0) 6 Y, 

X 1 1 2N z 


for some (r,q) 6 R x [L ] . Arguing as in [7], we see that r > 0. So r can 
be normalized to 1. Using a^ = j(x,u;X,v) and a2 = y(x,u) in (2.8), we get 


J(x,u;X,v) + Z <q.,b.> s iy(x,u) 

1 i x 


[nererore 


<b(x,u,q) ^ t(x,u) ; 


inf <*>(x,u,p) 2 0(x,u,q) 2 'j'(x,u) 

P 6[L 2 ] N 


Combining the above with (2.6), we conclude (2,7) 


Remark 2,5 It is well understood in duality theory that the "hyperplane" 
separating Y and Z will define and attain the optimal dual multipliers 
[?i (when ^(x,u) < °°) . □ 
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The arguments for the following lemma are the same as those for Lemmas 
2.3 and 2.4, the proofs are therefore omitted. 


Lemma 2.6 Assume that J(x,u;X,v) is concave with respect to (X,v) and 

IN 1 

convex with respect to (x,u) for (X,v) 6 [H ] x U, (x,u) € H x U, 

n n 

X(0) = X Q , x(0) = x Q . We have 


(2.9) 


sup 


inf 

(x,u) H^xu 
x(0)=Xq 


[t(x,u) + <p n ,(DE)>] = inf 

(x,u) 

feasible 


ty(x,u) ] . 


o 


Remark 2 . 7 In (2.4), we introduce N Lagrange multipliers p^, one for each 
player. In (2.9), we introduce the joint multiplier p^ commonly shared by 
all players. □ 


Proof of Theorem 2.1 From Lemmas 2 . 4 and 2.7, we conclude that 


(P) = inf sup {j(x,u;X,v) ] (x,u) and (X,v) are feasible) 
x,u X,u 


= inf [ sup J(x,u;X,v)] 
(x,u) (X, v) 

feasible feasible 


1 inf \J/(x,u) 
(x,u) 
feasible 


= sup inf [>jr(x,u) + <Pg f (DE)>] (by Lemma 2.6) 

P Q €L* (x,u)€H^xU 
x(0)=x q 
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UP 2 ln i “f N SUP i» [J(x > u 5 X » v > + 2 <p,(DE).>+ 

PQ a n ( x > u )^ n xu P^[l/] N (X,v)€[H^J N xu i=l 1 1 


x(0)=x / . 


XCO)^ 


+ <P 0 ,(DE)>] (by Lemma 2.4) 


= max min L(p,q) = (D) , (by Remark 2.5). 

ct 2 Pt 2 . N 

V L n p€[L n ] 

Hence if (x,u;X,v) is feasible and solves (P) and if (p Q ,p) is feasible 
and solves (D) , we have 

c = J(x,u;X,v) = min max J(x,u;X,v) 

(x,u) (X,v) 

feasible feasxble 

= max min L(p_,p) 

, T 2 , rT 2.N u 

p 0 a n pS[L n ] 

= L(p q ,p). 

So the proof is complete. □ 

There are still improvements on Theorem 2.1 that could be made, but that 
would make Theorem 2.1 unduly too general and lengthy, so we choose not to do 


them here. 
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§3. Linear Quadratic Problems and Synthesis 

From now on throughout the rest of the paper, we consider the linear 
quadratic problem whose cost functionals are given by 


(3.1) 


J^x.u) 


- I rT 


f/J [|C i (t)*(t)-« 1 (t)r k 


+ <M. (t)u. (t) ,u (t)> Jdt , 
li i m, 

H 1 


i = 1 N, 


(x,u) feasible. 


where we assume that C^(t) and M^(t) are matrix-valued functions of 

appropriate sizes and smoothness, z^(t) is a vector-valued function. 

2 2 

Furthermore, M.(t) induces a linear operator M . : L -*■ L which is 
i i m^ m_^ 

positive definite: 


(3.2) 


<M 1 u i ,u i > 


2 v 0 i|U ± | 


m. 


m. 


1 < i < N, 


for some > 0. 

The main objective of this section is to give a formal derivation of the 
adjoint equations and the Riccati equation from the dual formulation. Later on 
in §4 we will see that under certain sufficient conditions these procedures can 
be justified by Theorems 2.1 and 4.6. 

We use the definition of J(x,u;X,v) as in (1.8). For any feasible (pg,p), 
the Lagrangian L is 

N 

L(p ,p;x,u;X,v) = J(x,u;X,v) +<p_,(DE)> + 2 <p , (DE) > 

u u O 4*1 1 1 n 


(3.3) 
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N 

" 2 [J i (x,u 1 ,...,u N ) -J i (x 1 J u 1 ,...,u i _ 1 ,v i ,u i+1 ,... > u N )] 

i=l 


N 

+ <p_,x - Ax - 2 B.u, - f> 

0 i-l 1 1 ,2 

Li 

n 


N 

+ 2 <p, 

i-l 


. i , i 
,x - Ax 


- 2 B.u. 

j^l J J 


B.v, - f > 
x i 


n 


We first study max L(p n ,p;x,u;X,v) . Assume that for given p.,p,x,u, 
(X,v) U U 

x(0)=x 0 

A . 

the maximum is attained at (X,v). By a simple variational analysis on x , 
we have, necessarily, 


(3.4) -<C. (C.x'^-z.) jy'S* +<P.»y 1 - Ay X > = 0 , (C = adjoint of C) 

ill 2 l 2 * 

L L n 

n n 


for all y 1 € H*. y 1 (0) =0, 1 2 x 5 N. 


From variational analysis, we also have 


(3.5) Pi € H*, p± (T) = 0, 


and 


- <c 1 (C 1 x :L -Zi) + p ± + A*p 1 ,y i > = 0; 

L 

n 


1 s i < N. 


Hence 
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(3.6) ^ = -A* Pi - C^CC.x^). 


Similar variational analysis on gives 


<M 1 v.,w 1 > -<P 1 .B 1 w 1 > 

1 L 

m_^ n 


Vw t 6 K * 

1 m i 


(3.7) v. = -M“ 1 B*p i , 


1 < i < N. 


We now consider min L(p ,p ;x,u;X,v) . Assume that the minimum is 

(x,u) L 
x(0)=x Q 

attained at (x,u). By the same reasoning as above, we get 


(3.8) p Q € h£, Pq (T) = 0, 


(3.9) p Q - -A*p + X C*(C ) 

1=1 


- ^ 

(3.10) u = MT-B (p + 2 p ) 

U j=l 3 


M" lB *(Po+ p s- p i), 


P = 2 p. 

S J-l j 


Let L(Pq>p) be as defined in §2. If the problem max min L(pg,p) attains its 


P 0 p 


max-min at (p Q ,p), then p^ and p satisfy (3.8), (3.9) and (3.5), (3.6). 
Therefore we obtain X,v,x,u,p n ,p as the solution to the following two point 


boundary value problem: 



Theorem 3.1 Let &,v,x,u,pQ and p satisfy 


L(p ri >p) 3 max min L(p ,p) 


, T 2 

p 0 fL n p6[ V 


O’ 


A A A A 

= max min L (p 0 ,p;x,u;X,v) 
P 0 P 


= max min min max 

P n P x,u X,u 

x(0)=x 0 X(0 )=Xq 

A A A A 

= J (x,u;X,v) 


= min 
x,u 

x(0)=x Q 


max J(x,u;X,v) 
X,v 

X(0)=X Q 


Then x,& 



P 0 ,P - (P r 



L(p 0 ,p;x,u;X,v) 


are coupled through 
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(3.11) 


dt 


x 


~N 

x 


N 


r 


o 

N 


2 C*C. 
1=1 1 1 


0 

A 

0 

0 

* 

- c l c l 


0 

0 

A 

0 

0 


1 N 

s i s ir • • s in 


N 


S . .S._ T 
Nl NN 


-A 0. • • 0 
* 

0 -A 


-S C N ° 


-A 


n n 


x 


-N 

P 


f 

f 


i - 
* 

C 1 Z 1 


Vn 


x(0) = £ X (0) = .... = x N (0) = x 


0 


Pq (T) = P 1 (T) - ... - P n (T) = 0 , 


and u,v satisfy 


u. = M. 1 B*(p n + p - p.) 

1 X x *U *S X . 


^ -1 *- 

v, = -M . B . p . , 
i x x r x 


where in (3.11), 


N -1 * 
(3.12) S = Z B.M. B. 

j=l 3 3 3 » 


S = Z B.M?^* 
1 j*i 3 2 


2 s - (1 - “ B k M k lB i c » = Kronecker's delta). □ 


II MSS 
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Decoupling can be achieved by assuming the feedback affine relation 
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which denote, respectively, the first, second, third and fourth quadrant of 
blocks of matrices m the big matrix in (3.11). From (3.11) and (3.13), using 
the above notations, we get the Riccati equation 

P+PA+ A*r + p^p - c = 0 

(3.14) 

P(T) = 0, 

for P. We also have 


tx + (P5 + /A )nr + Pf - £ = 0, 

(3.15) - 

ir(T) = 0, 


where 



The reader may compare the Riccati equation (3.14) from our dual approach 
with that in [9, (4.30)] obtained form the primal approach or that in [6, p. 312, 
(8.5.23)]. 



26 


24. The Dual Max-Min Problem 


We study the dual problem in this section. This will become the basis of 
the finite element computations in §5. 

* N ^ 

Henceforth, for simplicity, we denote the operators C.C. and 2 C.C. 

1 1 i=l 1 1 

N i x 

£ "k 2 

(induced by the matrices C. (t)C, (t) and 2 C, (t)C.(t)) in L as 


i=l 


n 


1^(1 5 i - N) and Cq, respectively. 


We will need several assumptions as we proceed. First, we assume 

2 

(Al) each operator E^(l 5 i < N) is strictly positive definite in L n> 


From (3.6), we get 


(4.1) 


1 -1 . * * 

= (p. + A p. - C z ) 

X *1 *1 X X 


By (Al) , Eq is also strictly positive definite. By (3.9), we get 


(4.2) 


x 


1 * N * 

= E 0 (P Q + A p Q + 2 C « ) 

x=l 


We now substitute (4.1), (4.2), (3.7) and (3.10) into (3.3). Integrating by 
parts with respect to Pq and p^(l - i - N) once, using the end conditions 
(3.5) and (3.8) and simplifying, one obtains 


(4.3) L(p Q ,p) = L(p Q ,p;x,u;X,v) 


N 

“ “ \ <P 0 +A *Po’ C 0 1 ^0 +A * P 0' )> + 2‘ . Z < Pi +A *Pi» C i 1 (P i +A *P i ) > 


i=l 
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f <p 0 +p s .s(p 0 + p s )>+ <P 0 + P s . i f i B i M I lB Jp i > - < Po +A * I ’o>‘ : o X 


H s -1 * 

2 <p ± + A P 1 »E i c ± z ± > _ <P 0 + P s » f > “ <P 0 (°) + P s (°)>Xq> 


N N 

1 " * » " * , I,, „2 

o <c n C 2 C.z ), Z C z.> + ^ t! z fl 
u j-1 J 3 j-1 J J 


10 

2 IT., 
1-1 


, 2 N 2 

where ||z|| = 2 ||z.|| , and p ,S are defined as in §3. 

i=l 1 t 2 S 

He 

*i 


We are now faced with the ■ problem of max min L(pg,p). It 

PQ P 

is easy to see that L(Pq,p) is strictly concave in p^ for any given p. 
However, for any given p^, L(Pq,p) is not necessarily convex in p because 
of the negative sign in front of TT^* To circumvent this, we will need the 
following import assumption - : 

-1 2 

(A2) The positive definite operators E^ (1 S i S N) in L n are large 

enough so that 


N N 

(4.4) \ ^ <P i+ A’‘p 1 .C- l (p 1 +A% 1 )>-i< Ps ,S Ps >+ <p s .J i B^ 1 !^ 


N 

>v 1 z ||p i| 2 , 
i=l 1 


for all P i € Hg n = {q|q,q € L 2 ,q(T) » 0>, for some v 1 > 0. 

We remark that even if E^ , 1 2 i < N, are not large enough, the above 

assumption can still be valid provided that T is chosen sufficiently small, 
because in this case, the first positive definite quadratic form in (4.4) will 
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have a larger coercivity coefficient to bound the L -norm, when the interval 
[0,T] is small. This agrees with the assumption that t^ - t^ be sufficiently 
small in [ 9 , p. 114, line 15]. 


Another special case wherein (A2) holds without requiring 
1 S i S N to be large is when 


-1 


N = 2, U L = U 2 

B* = B 2 = B, for some B 2 0. 

It is easily seen that now 

(4.4) - j Z < P i + A' f p i> C i 1 (p. + a' < p i ) >--j -2 < p gJ Bp s > + < P s> Bp s > 
i=l ’ 


* 1 iJi* + ^V 1 ! 1 ( Pi * A V > • 


so (A2) holds. 


Remark 4.1 We believe that if (4.4) is not a positive semi-definite quadratic 
form, then J(x,u;X,v) is not convex in (x,u) (J is always concave in (X,v) for 
any given u), thus hindering the existence or uniqueness of the equilibrium 
strategy. This is still being investigated. a 

Because the quadratic form (4.4) is symmetric, using the end condition 
p i (T) =0 (1 5 i 2 N) and the Poincar 6 inequality, we see that for 

11 122 2 1 

P = (p^» • • • » P^) 9 P = (p-|^-'*>P N ) € t H 0iJ 9 the bilinear form 


N 

(4.5) P(p 1 ,p 2 ) a | <p* + A*pJ, «i 1 (Pi+A*pJ)> - ^ P J,SpJ> 


i -1 


11 *? - 1*2 1 2 N - 1*1 
+ 2 <p s’ * B i M i B i p i >+ 2 <P s^ B i M i B i p i > 


1 N 

defines an equivalent inner product in [Hq^J • 
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Lemma 4. 2 Under (Al) and (A2) , for each given Pq, L(pg,p) is strictly 

convex in p and for each given p, L(Pq,p) is strictly concave in Pq. 

Proof: For each given we can write L(pg,p) as 


L(pg,p) = P(p,p) + linear terms in p + constant terms (depending 

on p rt and z . ) . 

^0 l 


1 N 

Since (3 forms an equivalent inner product in [H^ ] , we conclude that 
L(Pq>p) is strictly convex in p. 

The second assertion is already clear. i 


For each given p^, L(pg,p) is strictly convex, continuous and coercive 
in p (i.e., L(p n ,p) -*■ + ” as ||p|j -*• + “>). Therefore 


(4.6) min L(p Q ,p) = L(p 0 ,p(p Q )) 

is uniquely attained at p(Pq)» depending on p Q . 

From a straightforward variational analysis (or the Euler-Lagrange equations) , 

we see that p(Pq) satisfies 


N 


.-1 


r ^ 1 ( Pi +A ‘ 5 i ) +s <p 0 + p s > - l B jVj g j 




-1 * d ,„-l * 


(4.7) 




P ± (T) - 0, 


"' 1 (0)[p i (0) +a"(0)p 1 (0)] = -x 0 + E i i (0)C' i (0)z i (0); 1 < i < N, 


,~ 1 . 


V 
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—X ^ 

where it is assumed that (1 5 i S N) are sufficiently smooth so that 

{C^ 1 (0)C*(0)z i (0) exist. 

Now, consider L(Pq) = L(pq,p(Pq)) as a functional of p^. It is easy 
to verify that L(Pq) is concave with respect to p^. In fact, we have 


Lemma 4.3 L(p Q ) is strictly concave with respect to p Q . 


12 1 

Proof: For any 0 f [0,1] and any p Q , p Q € H Qn> we have 


(4.8) L(0Pq + (1 - 9)Pq) = min UGpJ + (1 - 6 )Pq ,p) 

p^4/ 


- min {-e||[9pi+ (l-G)p^] +A [©pi + (1 - 0)p5] || 2 
crw 1 1 N t 2 

P* [H 0n ] L n 


+ [L(0 P q + (1-0) P q,p) +e|| [0 pJ + (1-0)Pq] + A* [0pJ + (1 - 0)p 2 ] [| 2 ]> 


n 


where in the above, e is chosen sufficiently small so that - -j (Cq + e I 
is still strictly negative definite. Continuing from (4.8), we get 


(4.8) = -e||(£ + A*)[e P J + (l-0)pJ]!| 2 + min {L(0pj + (1 - 0)p*,p) + e|| (~ + A*) 


2. ,,2 


t 2 tru 1 i N 
L n P 6[H 0n J 


[©Pq + (1 ~ © ) Pq ] II 2 } 

L 2 

a 


^ -e||(^ + A*)[0pJ + (l-0)pJ]||^ 

L 

n 


•1 . * 1„2 


+ min i0[L(pJ,p) +e||pJ+A"pjn + (1 - 0) [L(pJ,p) + slip* + A* P 2 || 2 ] } 

r tt! t N 


■’'“W 
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because the parenthesized term is concave and because 
negative definite. 

(continuing from the above) 


- j E" 1 + S’ I is 


(4.9) 2: - s|i(-^- + A*) [0Pq + (1 - 0)pq]|| 2 + 0 min {L(pj!j,p) + e||pg +A*Pg|| 2 } 


t 2 txu 1 ! N 

L n p€[H 0n ] 


+ (1 - 9) min (L(Pq,p) + e||p 2 + A*p 2 || 2 }. 

^ 4 / 


If Pq # Pq and 0 t 0,1, then 


-e||(^ + A*)[0 P J + (l-0)p 2 ]|| 2 + 0e||pJ+A%j!! 2 + (l-9) S i|p 2 +A*p 2 !| 2 > 0, 


so (4.8) and (4.9) give 


L(0p^ + (1 - 0)pjb > 9 min L(p^,p) + (1 - 0) min L(p 2 ,p) 
u u 1N U 1N U 


= 0L( P q) + (l-0)L(p 2 ), 


proving strict concavity. 

We proceed to study max L(p ) . 

p.< 


Lemma 4.4 Under (Al) and (A2) , L(p Q ) is (negatively) coercive with respect 


to p Q , i.e., 


L(p q ) -*■ “ " as ||p 0 | 


-*■+". 


h; 


On 
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Proof: Because 0 € [H^ ]^. 

*• On ’ 


we have 


(4.10) l (Pq) = min L(p Q ,p) < L(p Q ,0) 


1 it ^ 2_ 

= - 2<P 0 +A P 0 ^0 ( P0 +A 

L R 

n 

? c jV + i N|2 - < p 0 +A \-s 1 ” v ± > 

i 1 i=l 


We use 


! < %’ f> 2l “ 2^ P d 2 + 2s^ 2 ~ 2*^0 +A P 0 ,!C 0 (p 0 +A P 0 )> + 2^ f ^> 

L Li L 

n n n 

Mo +a * ! ’o’ i o 1 , z , c I*i>l s f <Po +A ‘ p o- ,: o 1< Po +A V > + ”, Vi-'o 1 

i-i " 1=1 i * 


l< p 0 (0) 'V.nl 5 f» p 0 <0 )»!n + ^“fn S 2 K<p 0 +A4p 0’ I 0 1(p 0 +A % )> 


R 


+ 2e II x o^ 


in (4.10); in the above the constant K > 0 depends on Iq"*" only. Choose 
s sufficiently small. One sees that 


L(p 0 ) — L(p q » 0) 2 (_ L+eK+f^pQ+A^Q.C- 1 ^ 


•k 2. 

+ A Pn)’ - 7 <P 


0 ,Sp 0 > 


+ + ^l £ " 2 < 



» 


the right hand side tends to 


as 


II Pr 


h; 


On 


□ 


II M 
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The first main theorem in this section is 


Theorem 4.5 (Dual Saddle Point Theorem) Under (Al) , (A2) , the max-min 

problem max min L(pg,p) has a unique solution (pg,p). Furthermore, 

P 0 P 

(4.11) max min L(p~,p) = min max L(p ,p) . 

, U 1 , r „l -iN U , r „l ,N ,„1 

*0%n P^On 1 P^V p 0%n 

Proof : We use the standard saddle point argument [5], except that we replace 

the compactness condition by coercivity. 

For each Pq, there exists a unique P(Pq) minimizing L(pg,p) with 
respect to p as in (4.6). 

By Lemmas 4.3 and 4.4, max L(p Q ) = L(p Q ,p(p 0 )) also has a unique 

Po< 

minimizer Pq. Hence max min L(pg,p) has a unique solution (p ,p) (with 

P 0 P 

P = P(P Q )): 

(4.12) L(p_) = max L(p ,p(p )) = max min L(p n ,p) = min L(p ,p). 

cw 1 rrw 1 i N , r „l iN ° 

p 0 ffl 0n Po^On p€| - H 0n^ 

1 IN 

For any p Q € H Qn , p 6 [H Qn ] ant ^ 0 € (0,1), we have 
L((l - 9)pg + 9p Q ,p) > (1 -0)L(p Q ,p) + 9L(p 0 ,p) 

> (l-0)L(p n ) +9L(p n ,p) . 
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In particular, we choose p = p((l-8)pQ+ 0 Pq) . From the above we get 
L(p q ) >L((l-0)p o + 0p o ) > (l-8)L(p 0 )+ 8L(p o ,p((l-0)^ o +8p o )). 

Hence 

L(p q ) > L(p o ,p((l-0)p o + 0p Q )). 

Noting that p((l-0)p + Sp^) is continuous with respect to 0, one lets 8 

tend to 0+ and gets 

L(p 0 } > L( Pq , p(p Q ) ) , Vp Q 8 HQ n . 

On the other hand, from (4.12), 

L(p Q ) 5 L(p 0 ,q), Vq 0 [ H J n 3 N - 

Therefore we conclude 

L(p 0 ,p(p 0 )) = L(p q ,p) 2 L(p q ) = L(p 0 ,p) < L(p 0 ,p),Vp,p 0 . 

Hence (4.11) is proved. □ 

So far, our derivation of the dual problem is only formal because we have 
not yet verified the assumptions in Theorem 2.1 that J(x,u,X,v) is convex 
in (x,u) and concave in (X,v) and that inf sup J(x,u,X,v) is attainable. 
These questions are answered in the following theorem. 
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Theorem 4.6 (Primal-Dual Equivalence Theorem) 

Assume that C^(t), z^(t), 1 S i S N , f(t) and 1 - 1 < M, are 

sufficiently smooth (as functions and operators, respectively). Under assumptions 
(Al) and (A2), for the linear quadratic differential game (0.1) and (3.1), let 


J(x,u;X,v) be defined as in (1.8). Then 


i) 

J(x,ujX,v) is 

convex in (x,u) and strictly concave in (X,v); 

ii) 

A A 

there exist unique (x,u 

) and (X,v) such 

that 


(4.13) 

inf 

sup 

J(x,u;X,v) = 

min 

max J(x,u;X,v 


(x,u) 

(X,v) 


(x,u) 

(X,v) 


feasible 

feasible 

feasible 

feasible 


A A A 

= J(x,u;X, 

~ \ 

v) < oo 

9 



iii) 






(4.14) 

min max 

J(x,u 5 ; 

X,v) = max rain J(x,u 

> X , v ) 


(x , u) (X,v) 


(X,v) (x 

,u) 


iv) 






(4.15) 

L(p 0 ,p) = 

max 

min L ( Pq , p ) 




»0 a n 

. r . 2.N 




= 

max 

„2 

p 0 a n 

min min 

D€ r L 2 l N < x > u ) 

P 1 n J x(0)=x Q 

max 

(X,v) 

X(0)=X Q 

UPq»P;x,u;X,v) • 


_ 

min 

max J(x,u 

>X,v) . 




(x,u) 

(X,v) 





feasible feasible 



v) The 

(second) dual 

of the (first) dual problem (namely 

, (D)), obtained by 

h ' 

■ h ■ 0 (1 

5 i i N) 

as constraints in 

L, recoveres to the primal 


problem (P). 



36 


Proof: The proof is based upon the "ref lexivity" argument that " the dual 

of the dual is primal ". 


By Theorem 4.5, L(pg,p) attain its unique saddle point at (Pq,p). 

In finding the saddle point of L(pg,p), we regard ~ p^ = 0, 

0 < i i N , as constraints and introduce Lagrange multipliers X^, X=(X^,. 



and consider 


inf 

X n l2 

0 n 


sup 
2 N 


X L 


n 


sup 

p 0’^0 

P 0 <T)=0 


inf 

P.P 

p(T)=0 


Kp 0 ,p 0 ,p,p;X 0 ,X) 


where 

(4.16) 


d JM . 

i Cp 0 »p 0 > p,p;X 0 ,x) = [l(p 0 ,p 0 , p ,p) + <VV at V + .^i^i -ft p i>l' 


and L(p 0 ,Pq,p,p) is the same as that in (4.3) except that we now regard 
Pq and Pq as unrelated . 

Define 

(4.17) I(X 0 ,X) = sup inf I(p 0 ,P 0 ,P,p;X 0 ,X) 

P 0 »P 0 P«P 

p Q (T)=0 p(T)=0 


We now apply (the proof of) Theorem 2.1 to L(p Q ,p), subject to constraints 
P i - ^jr p^ = 0, p^(T)=0, 0 2 i < n. It is easy to see that all the assumptions 
of Theorem 2.1 are satisfied by L(pg,p), since by (Al) and (A2), L(pQ,p) is 
strictly convex in p and strictly concave in p^. So we have a unique 

A A 

€ L x [ L such that 
u n n 
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I ( Ag , A) 3 min max 


V L n 


KAq.A) 


= min max max min 

x o x p o ,p o p,f * 

P 0 (T)=0 p(T)=0 


1 (p o ,p o* p,p;X o’ x) 


max 

p 0 

P 0 (l)-0 


mm 

P 

p(T)=0 


L(p 0 ,p) 


L(P 0 »P). 


On the other hand, from (4.16) and (4.17), by variational analysis on the 

• • # # 1 r 1 1 N 

Pq, Pq , p, p variables, we have, necessarily, that Aq « A 6 [H^J 

and 


(4.18) A n - I n A (p ft + A Pn + z C, z.) = 0, 

i= 


k 0 “o ^0 ■ “ 'o .- 1 J J 


(4.19) 


N 


x 0 ' [AC 0 1(p 0 + A * p 0 + C j z j } ■ S(p 0 + P s } + V^Vi + f] 


(4.20) A. + E? 1 ^. + A*p. - C*z.) = 0 

l i l l li 


(4.21) 


• — X • it N ^ N — i ^ 

A. +[ AC. (p. + A p. - l C.z. ) + S(p n + p ) - £ B .M . B .p . 

1 11 1 j =1 J J "0 *s ."j_ J J J*J 


B.M7 1 B^p. - f] =0 

l l 1*1 t 


A - 1 2 N. 

In the above, p Q , p Q , p, p depend on \ Q , Now define n 3 (t^,...,^) 
and C 3 (C 1 ,...,? N ) by 
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(4.22) = m7 1 B*(p 0 + Pg - p. ) , l<i<N, 

(4.23) q E -M7 1 B*p i , liiSN, 


From (3 . 12) » (4.18) , (4.19) and (4.22), we see that X^ satisfies 


(4.24) 


X. = 


N -i * 

AX + S(p n + p ) - E B.M.B.p. +f 
0 ° 3 j=l J 2 33 


N . -1 * 

= AX q + ^ B.[M. B. (p Q + p s - p.)] + f 


N 

= aX_ + 2 B-n- + f. 

0 j-1 J i 


Similarly, from (3.12), (4.20) - (4.23), we get 


(4.25) 


X. = AX. 
i l 


L B n. + B.£ . + f. 

jH J J 11 


The initial conditions satisfied by Xg, X, (1 - i i S) are just 
(4.26) X Q (0) = x QJ X i (0)=x Q , l<i<N. 

This can be easily verified (e.g., by comparing (4.18) with (4.7.4)). 

Substituting (4.18), (4.20), (4.22) and (4.23) into L(pg,p), we get 

I(Xq, X) 3 I(X 0 ,n;X,c), Which is convex in (Xg,n) and concave in (X,?). But 

this I(X 0 ,n;X,s) is just J(x,u;X,v) through identifying (X Q ,ri,X,^) with 
(x,u;X,v), subject to (4.24) - (4.26), i.e., subject to (1.10)=0 and 
(l.ll)-O (0 < i < N) 

J(x,u;X,v) is convex in (x,u) and concave in (X,v) because 
J(x,u;X,v) = I(X Q ,ri;X,5), which is the dual of L(p Q ,p) which is convex in 

p Q and concave in p. The fact that J(x,u;X,v) is strictly concave in 

(X,v) for any given (x,u) can be verified directly from J itself. 

The min-max and max-min in (4.14) are exchangeable because of (4.11) 
in Theorem 4.5 . 


0 
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Theorem 4.7 (Existence and Uniqueness of Equilibrium Strategy for N-person 
Linear-Quadratic Differential Games) 

Assume that (Al) and (A2) hold. Then the unique saddle point 
(x,u;X,v) of (4.14) satisfies the property that u = v and x 1 • x on 
[0,T], where x 1 is the i-th component of X. 

Thus 

(4.27) J(x,u;X,v) = min max J(x,u;X,v) = max min J(x,u;X,v) = 0, 

(x,u) (X,v) (X,v) (x,u) 

so u is the unique equilibrium strategy for the N-person differential game. 

Proof By (4.14), the saddle point property for J is uniquely satisfied 

A A A A 

by (x,u;X,v), so we have 

min max J(x,u;X,v) = max J(x,u;X,v). 

(x , u) (X,v) (X,v) 

feasible feasible 

•• * .A A A a 

Since the RHS above is uniquely attained by (X,v) (X depends on both u and 

/v 

v), we see that v is uniquely characterized by 

A 

v.=v. 

1 1 

where 3 denotes the Frechet derivative with respect to v.. 
v. v 1 

i 

Similarly, we have 

min max J(x,u;X,v) = min J(x,u;X(u,v) ,v) 

(x,u) (X,v) (x,u) 

where in the RHS above X(u,v) = (x^(u,v) , . . . ,x N (u,v)) depends on u and v 


(4.28) 


3 v J(x,u;X,v) i ^ = -3 v> J i (x 1 ,u 1 ,...,u i _ 1 ,v i ,u i+1 ,...G N ) 


V=v 


x L = Ax 1 + 2 B.u. + B.v. + f 

ji“i Jj 11 


x 1 (0) = Xq. 


as follows: 



A 

Thus u is uniquely characterized by 


(4.29) 


~/\/\ ^ - ~ i ^ 

) J(x,u;X(u,v) ,v) = E 3 jj.(x,u, . . . ,ulJ - J . (x J (u,v) ,u. , 
u i j=l u i J « J x 


V . ,u. , , . . .«,)] 

J j+1* N J 

=0 at u = u, for i 


*“ 1 , . . . ,N. 


Therefore (4.29) gives 

(4.30) 3 J(x,u' L ;X(u 1 ,v) ,v) | 


u. 

1 


- o. 


u. -u. 
1 1 


where 


= < V 


• jU# i jU* jU. . ) 
1-1 1 1 + 1 ’ 


,,u N ), for i = 1,...,N. 


But, evaluating the RHS of (4.29) with u = 


u , at u. = u. , we find that 
i l 


) [j.(x,G 1 ,...u._i,u i ,u i+ i,... ,u H ) “ Jj(x J (u 1 ,v),u 1 , . . .u i _ 1 ,u i ,u i+1 , . . 

i J 

u j+l>* • • | 0> 

J * 11 . =11 . 


A . A 


u . =u . 

1 1 


if j t i- 

So (4.30) is reduced to 


(4.31) 3 u [j j (x,G 1 ,...a._ 1 ,u.,u. +1 ,..,u N ) - J i (x J (u 1 ,v),G 1> ...,G i _ l ,v i ,G. +1 , 


= 3 u. J i (x,u i’*'” u i-l ,u i’ U i+l , "* u N ) I °* 


u. =u. 
i l 


because the second term in the above bracket is just a constant. 

Comparing (4.28) with (4.31), we see that u^ and v^, i = 1,...,N, 
satisfy the very same equations, whose solutions are unique. Hence u = 
is proved. 


< > 
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Because u - v, we conclude immediately that x 1 = x,\/i = 1,...,N and 
that the saddle point value (4.27) is 0. So u is an equilibrium strategy. D 

Remark 4 . 8 ~~ The above' theorem says that,' under "(Al) and (A2), any N-person 
non zero-sum linear quadratic differential game is, indeed, a 2H-person zero- 
sum game , with N authentic players represented by u^, i < i < and N 
fictitious players represented by v^, 1 i i S N. □ 
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Since the validity of assumptions (Al) and (A2) is not affected by u* , we see 

8 

that all the theorems in this section, except Corollary 4.7, remain valid for 
problem (4.32). The result u = v still holds for problem (4.32). But, now 
x 1 t x in general, so the saddle point value of (4.32) is not equal to 0 in 
general . D 

Remark 4.10 For linear-quadratic N-person games, under (Al) and (A2), the 
Hamiltonian (1.18) and the Bellman-Hamilton-Jacobi equation must be at a 
saddle point (instead of just min-max) for all t or T € [0,T]. □ 
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1 5 Global Existence and Uniqueness of Solutions for the Riccati Equation 
The system of Riccati equations [8, (4.30)1 in Lukes and Russell's 
approach has been known to have only local existence and uniqueness of 
solutions. However, under our approach, we can prove that our Riccati equation 
has global existence and uniqueness of solutions. The proof is an extension 
of the control theory case, cf. e.g. [12, pp. 197-205], to our equation. 
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Further, let 

N 

v. 3 - 4 ) J (x , u ;X,v ; , . . . , ; tg , t ^ ) = E ~~ ^ ^ > u ^ » • • • » u ^_^ > 

i=l 


V i ,U i+l’ * ' * ,U N‘^i ;t O’ C l^ 


subject to (5.2) and (5.3). 


Lemma 5.2 Assume (Al), (A2). Let (x,u:£,v) satisfy (5.2) and (5.3) with tg = 0, 
t 1 = T, £ q = = . . . = = x Q . Let q Q , q 1 ,...,q N be the solution on [0 ,t] of 


(5.5) 


% = " A q 0 + . 2 n C i X ’ 

1=1 


q 0 (T) = o . 


(5.6) 


• « / 

q . = -A q . - E . i 

H i n i l 


q £ (T) = 0, 


1 < i < N. 


Then (x,u;X,v) is the unique saddle point for min max J(x, u;X,v,x ,X ;0,T) 

(x,u) (X,v) ° ° 

if and only if 


(5.7) 


u.=M: 1 B*(q 0+ Z q ) 
J5*l 


, 1 < i < N 


(5.8) 


-V -1 * 

v. = - M. B.q. 
l l li 


, 1 < i < N. 


Proof of Lemma 5.2 By Theorem 4.7, min max J(x,u;X,v;x„,X„;0,T) has a unique 

(x,u) (X,v) ° 0 

saddle point. If (x,u;X,v) is this saddle point, it is characterized by 


(5.9) J(x,u;X,v;Xg,Xg ;0,T) < J(xtex;u+eu;X+eX,v,XgXg ;0,T) , ye € R, 


(5.10) J(x,u;X,v;xg,Xg;0,T) > J(x,u;St+eX,v+ev;Xg,Xg;0,T), ye € H, 

where (x,u) and (X,v) and X (= (x 1 } x 2 , . . . ,x N ) satisfy 


x = Ax + Z B.u. 

• li 
i 

x(0) = 0, 


[o,t], 
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(5.12) 


(5.13) 


* • • 
x 1 “ Ax 1 + 

E B .u. + B.v. 

j*i J J 11 

on 

[0,1] 

XI 

/■N 

O 

It 

O 


1 < 

i < N. 

-i -i 

x ** Ax + 

2 B.u. 
jj*i J J 

on 

[0,T] , 

x l (0) = 0 , 



1 1 i 


Note that in the RHS of (5.9) X +eX appears because it is also dependent on 
u + eu. 

From (5.9), we get 


which is 
(5.14) 


— J(x + ex, u + eu; & + eX, v; x Q , X Q ; 0, T)| £ _ Q = 0, 


N T 

22/ [<C. (t)x(t),C. (t)x(t)> + <M. (t)u. (t),u. (t)> 
i = l 0 ^ ^ 3. L 1 


- <C^(t)x(t) ,C^(t)x : ‘'(t)>] dt = 0 


From (5.5), (5.6), (5.11), (5.12) and (5.13), we have 

0 = <x(T),q 0 (T)> + 2 <x 1 (T),q i (T)> 

i 

T 

= < x( 0 ),q Q ( 0 > + 2 <x i ( 0 ),q i ( 0 ) > +/ ^ [< x(t) ,q Q (t)> + 2 < x^t) ,q. (t)>] dt 

i 0 i 

T . . . 

= / [<5(t),q 0 (t)> + <5(t),q 0 (t)> + E < x 1 (t),q i (t)> + ? 4 ^(t) ,q. (t) >] 

T * 

= / [<A(t)x(t) + E B. (t)u. (t),q_(t) > +< x(t) ,-A (t)q n (t) + E ®.(t)x(t) > 

0 .liu u .x 


dt 


+ 2 < A(t)x X (t) + E B.(t)u.(t),q. (t) > + E <x L (t) ,-A (t)q (t) - (C. (t)x 1 (t)>] dt 
i j^i J J 1 i i 1 
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T . 

(5.15) - 2 / [< C i (t)x(t) # C i (t)x(t)> + <M i (t)u i (t),U i (t)> - <C i (t)x 1 (t),C i (t)x 1 (t)>]dt 

i 0 


1 ^ ^ 

+ 2 / [-<M. (t)u. (t),u. (t)> + <B. (t)q_(t) ,u- (t)> + £ <B. (t)q. (t) ,u. ( t ) >1 dt. 

i 0 1 1 1 ° 3 


Comparing (5.15) with (5.14), we see that (5.9) holds if and only if 
T 

S [ _ <M- (t)u. (t) ,u. (t)> +< B. (t)q n (t),u. (t)> + z <B. (t)q.(t),u. (t)>]dt = 0 

Q X X 1 X U X j^X ^ J X 


for all € U^, i = 1,2,...,N. This gives 

* 

-H.u. + B.q rt + 2 B.q. =0, 

l l i M 0 i H J 


U i i N, 


which are just (5.7). 

We can obtain (5.8) in a similar manner. The proof of Lemma 5.2 is complete. 

The proof of Lemma 5.2 indicates that with appropriate simple adaptation, the 
arguments given in [l2 ,pp . 197-205] are immediately applicable to our proof. As in 
[l2 ,p . 199 , (2. 16)] , analogously, we now claim that we have 

N 

(5.16) q n (T)S(T) + 2 q.(T)x 1 (T) = min max J(x,u;X,v;x(x) ,X(t) ;t,T) , x€[0, 

U i-1 (x,u) (X,v) 


where (x,u;X,v) solves the min-raax problem, on the RHS above with (arbitrary) 
initial condition (x(x),X(x)) for (x,X) at the beginning time x. 

Because f and £ in (3.14) are 0, the solution Y of (3.14) is also 0 on 
[0,T]. Thus, by (3.12), 



q o (t) * 


X ( t ) 

(5.17) 


= IP (t) 



q(t) 

ft m 


X(t) 

■ 


t 6 [0 ,t], if ® exists. 


From (5.16) and (5.17), we get 




P * 

x(x) 


r 

x(x) 

(5.18) 

< IP(x) 

A 


A 



X(x) 

w 

9 

X(X) 

L « 

whenever IP 

exists on 

[x,T]. 



min max J(x,u;X,v ;x(x) ,X(x ) ;x ,T) , 
(x , u ) (X,v) 
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The nonlinearity in. the Riccati equation (3.13) satisfies the local 
Lipschitz condition. So, by the Picard local existence and uniqueness theorem, 
the solution IP (t) of (3.14) exists at least on a half open interval (t',t], 
for some x'<T. Assume the contrary that IP does not exist globally on [o,t]. 
Then there is at least one x 1 € [0,T) such that 

lim | jP(t) | | = oo . 

t+T' 

This means that there exists at least one (xq,Xq,Xq, . . . ,Xq) € [lR n ] N+ ^ such 
that 
(5.19) 

(5.19) lim | < IP (t) 

t|X ' 

But, if we choose t Q = x ' , = x Q , = x^ (1 < i i N) in (5.2), (5.3) and 

apply (5.18) and Remark 4.8, we see that 

X 0 
1 

X 0 

N 
X 0 

= a finite number, 

contradicting (5.19). 

Therefore IP exists uniquely on [0,T^. 


> = min max J(x,u;X,v;x - n ,(xi, . . . f xJJ) ;x' ,T) 

„Uy ..'i u 0 0 





§6 . The Dual Variational Problem and Finite Element Approximations 


Let F: x "*■ 1R be a real-valued Frechet differentiable mapping from 

a product Hilbert space H^ x H 2 into 3R . Assume that F(x,y) is strictly 

convex in x (for each y) and strictly concave in y (for each x), and that 

(x,y) is the unique saddle point of F satisfying 

min max F(x,y) = max min F(x,y). 

xS^ y€H 2 y£H 2 XS^ 

Then xt can be easily shown that (x,y ) is uniquely characterized by 
(6.1) 3 x F(x,y) i x= - = 0 , 


( 6 . 2 ) 


3 F(x,y) | ~ = 0 , 

y > j \ y=sy * 


We now apply the above property to L(pg,p). It is easy to see from the 

theory in §4 that all of the assumptions above are satisfied. Therefore 

(PqjP)j the unique solution of max min L(pg,p) in x is 

P 0 P 

characterized by 3^ L(p^,p) = 0, 3 p L(pg,p) = 0. 

From (4.3), by a simple calculation, we get 


N 

(6.3) L(p Q ,p)-r = -<p Q +A*p 0 ,E 0 1 (r +A*r)> -<p Q +p s ,Sr>+<r,2 B^M'V’p^ 


* N 

-<r+A r,!-^ 2 C.z^> -<r,f> -<r(0) ,x n > = 0, Vr € , 

u ^ l i u Un 


(6.4) 


SpL(p 0 ,p) -s = 2 <p i +A*p 1 ,C~ 1 (s i +A*s i )> -<p Q +P g ,S 2 


N i * N N 

+ <P 0 + V l B i M i B i s i > + l B i M i^ 1 P i > 


N . * -1 * N N 

2 <s i +A - <Zs^,f> - <2 s^(0) ,Xg> = 0, 


V 6 IH 0r/' 
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The above two relations induce a bilinear form on x [Hg n J N : for r\ 

2 . „1 ,1.1 1, 2 , 2 2. r «l iN 

r € H Qn and s = s - 6 [H Qn ] , 


(6.5) a ( 


r ii 


r 2 i 

r 


r 

1 

* 

2 

_s _ 


_s _ 


. ^.1 , * 1 _-l/ • 2 * 2. 

) = — <r + A r ,E q (r +A r )> 


N N 

1 12 2 -1*1 
- <r + 2 st,Sr > + <r , 2 B.M, B.s7> 

j-1 J l 1111 

^ 1 * 119*2 1^1 ^ 2 

+ 2 <st+A st,E/(s:+A s,)> - <r + 2 s,,S 2 s,> 

i * 111 1 j-i J j-i J 


i. ; i N 


- 1*2 
r -*-t> _ 


N n N 


-1 * 1 


+ <r + 2 st, 2 B.M, B.s.> + < 2 s. > Z B.M B s,> 
1=1 1 1=1 1 1 1 1 1=1 1 1=1 1 1 1 1 ' 


and a linear form 0: for r 6 and s = 6 Hg n , 


( 6 . 6 ) 0 ( 


N 


N 


N 

A — I " A 

■ t r* 


) = <r +2 s, ,f> + <r(0) + 2 s. (0) ,x n > + <r +A r,E n 2 C,z.> 
1 J 1 JU u 1 


+ 2 <s i +A C^z^ 


Thus (6.3) and (6.4) are equivalent to 


(6.7) a( 





P 0 


■ r‘ 

A 



P 

km -A 


. s- 


) - 0 ( 


fr 


L s. 


) , V(r.s) 6 H J n x [hJ/. 
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We are now in a position to compute (Pg.p) by the finite element method. 

2 S 

As in [1], we say that c (0,T) is a (t 1 ,t 2 )-system (t^,t 2 are n °n- • 

k 

negative integers) if for all v 6 H^(0,T), there exists such that 


( 6 . 8 ) 


|v - v || < KhHIvll , VO < r) < min(k,t_), r) € K, 

H « Hp 


where [i = min(t 1 -r),k -rj) and K>0 is independent of h and v. 
Let c. be a Or, 1) -system. We consider 


(6.9) max min L(p Q ,p). 

p 0 ss h P«[S. 1 ] N 


It is easy to see that under (Al) , (A2) , there exists a unique saddle point 
(Pgh’Ph^ * S h X ^ S h-* N Such that 


L( ^0h’Ph ) = max min L(p q ,p), 

Po €s h 


This point (Pq^jP^) is characterized as the solution to the variational 


(6.10) a( «. 


P 0h r h 


P J ' > = 9C s ) . 6 s h * tv”- 


^ t(p } , t'r f } are basis for S, , [S ]^, respectively, then 

i=l i=i n n 


(6.x0) is a matrix equation = 0^, where 
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[ Vij 


1 

1 


— 1 
< 
t_k 

1 

9- 

1 

* 

_,JJ 




) 


1 2 i, j < (N + 1)J, 


1 2 j < (N + 1) J . 


More specifically. 


<t i + aV , it" 1 (f> + A V } : 


. N . N . * 

f< 2 VS'^> + c^, 2 \\\\' 
k=l k=l 




“h" 


i N j ± N - 1*1 

■<r,S 2 <P k > + <r, 2 

k=l k=l 


N 

2 +A (p i ,iE i 1 ((p^+A (p^)> - 
k=l ~ lV ^ 

N . N 
- < 2 < f S 2 cp 3 > 

k-1 k=l k 


N i N -1 * j 

+ < 2 cp k> 2 B k M k 'a k (p k > + 
k=l k=l 


^ i ^ -1 * i 

4< 2 cp£, 2 B m B cp > 

k-1 k-1 


* 

»f> + (0) ,x n > 


• 1 * i -1 7 

+ <\)r +A f,C n 2 C t z,_> 

k-1 


N 


-0 ,.Z, k k 


9 h = 


N . N 

< 2 cp k> f> + < 2 cp^(0),x Q > 

k=l k=l 


^ . i * i -l * 

+ 2 «p^.+A cpJ,(C k C k z k > 

k _i 


Note that 




is symmetric but non-positive definite. 
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Numerical analysis for general quadratic saddle point problems seems to be 
difficult. To make the above computations amenable to standard finite element 
error analysis, »once again, we need two more assumptionst 

(A3) the bilinear form a satisfies 



and 


(A4) the spaces {S^}^ satisfy 
inf sup 





for some y > 0,Vh > 0. 


The fact that the above two assumptions are realistic can be seen from 
the following 

Proposition 6.1 If i = 0,1,..., N, as positive definite operators, are 

-1 * 

comparatively larger than S and B^M^ , i = 1,...,N, then (A3) and (A4) 
are valid. - 

Pro ° f For any given (r 2 ,s 2 ) € Hj n x (or, (r 2 ,s 2 ) € S h x tS h ] N ), we 

have 
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2 [<r^ + A*r^,!Eg^(r2 +A*r^)> + 2 <s^ + A*s^,C i ^(s£H A*s^)> + <r^,Sr^>] 

^22 2^ -12 2^2^2 
-[<2 sT.Sr > - <r , 2 B.m/b*sS + <-r + 2 s,,S 2 s > 

j-1 j 1-1 1 1 1 1 j-1 2 j-1 2 


9 « o « -12 '2 -1 2 

+ <r - 2 3j f 2 B.M d*Sj> - < 2 a., 2 B M l*s >]. 
1=1 i-1 1 1 1 1 1=1 i=l 


If (i = 0,...,N) are large enough, the second bracketed term above can 

be at most equal to a fraction of the first bracketed term, thus for some 
X: 0 < X < 1, 


sup 

r 11 

,s L J 


Therefore 


= 1 


a( 


" 2 
r 


2 

s J 


5 X • the first bracketed term in (6.11) 


5 Y > 0, for some y* 


inf 


sup 


'r 


_ 

r 

2 

11= 1 II 

1 

Ls J 


- s J 



r 


r 2 ~ 



r 


r 


a( 

1 

9 

2 

) 


s 


s 



> Y > 0. 


Hence (A3) and (A4) are justifiable under the assumption. In fact, the above 
argument shows that assumptions (A3) and (A4) are related to the earlier 
assumption (A2). 


Theorem 6 .2 Let (PQh’^h^ t ^ ie s °l ut l° n °f (6.9) and let be a 

(tt, 1) -system. Assume that C^(t),z^(t), i = 1,...,N are sufficiently smooth. 
Under (A1)-(A4), we have 
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(6.12) 


IJo-W, + »5-5h« . = nf*tlp 0 l ♦ II5I1 1 

»l E«O n J N < tH a> 


(6.13) 


a A A 


'Po - Poh“, 2 + lip • p h» , „ - “^tllP'o'b + "p" , _ 

L r _ 2.N H 

n [L ] n [H ] 

n n 


1 9 1 9 fJ 

provided (Pq,p) € [Hg n fl H^] x [H Qn fl H n ] , where p. = min (t- 1,£-1) and 


. A A 


> 0 is a constant independent of (pQ,p). Consequently, 


(6.14) 


|L(p Q ,p) - L(p h ,p )| 2 K 2 h 2 ^[l|p II 2 + I!p|| 2 ] 

V [H e ] N 

n n 


holds for some K 2 > 0 independent of (p^.p) 


Proof : Because (p 0h ,p h ) satisfies (6.10) and (Pq,p) satisfies (6.7), we get 


a( 


A A 

p 0 - P 0h 

A A 

P " Pt, 


L h J 


N 


) - 0 . V(r h .3 h ) e S h x ts h ] . 


Therefore ([l,p. 186]) by (A3) and (A4) , one gets 


"<Po ■ f’oh’ P ' V® X . r.,1 ,N 5 (1 + •? inf 


H 0 n * [I W 


for some C > 0 independent of h. 
Using (6.8), we get (6.12) • 


‘V 8 ^ cs h* [s h ! 


N 


fll #o - r h» .1 


H 


On 


+ "P - S h" „1 ,N 1 




To prove (6.13), we use Nitsche's trick ([4], [10]). By (A3) and [1], for any 
g 6 L 2 x [L 2 ] N , we have a unique w(g) € x [H^]^ such that 
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a(»(g),y) - <g,y> 2 2,N • V ? € “on * tH On ,S ' 

L „ * 

10 M 

Furthermore, we have w(g) € [Hg n fl H n J x [Hg n Dll^] , provided that C^(t) 

and z^t), i = 1,2,..., N, are sufficiently smooth. (This w(g) can be 

obtained explicitly from integration by parts and it satisfies an equation 
similar to (4.7)). It is not difficult to verify that 


|w(g)|| ? , N - K'Hgl! , 2,N ’ 


H“ x [H ] 
n n J 


L x [L ] 
n n J 


where K' Is independent of g. By the very same proof of the Aubin-Nitsche 
lemma [4, p. 137], which remains valid under (A3) and (A4) , we get 


(6.15) 


- if-y i« s ch WoV + w « H i- 

n n J n n J 


sup 

n n 


[ rar 


«h €s h x ' s h’ 


N 


|w(g) - ^ h ||] 


But, by (6.8), 


UgfT 


inf 


^ xtL n ,N W'V 


N IWs) ‘ K bf ] -llif ' K' h ll”W» H 2 


S -ir— rr * K" • h • K'||g|| = K'K~h, for some K~ > 0 independent of g and w(g) 
llgll 


Using the above in (6.15), we get (6.13). 
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To show (6.14), we note that 



The first term on the right above is zero because of (6.7). The second term 
on the right can be estimated by using (6.12). Hence we get (6.14). □ 



( 6 . 20 ) 
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( 6 . 21 ) 


- 

n 


n 


< K 3 h' 




tllPo" e + 

H 

n 


'PH l N 

' H n> 


]. 


for some > 0 independent of x, u, v, Pq and p. □ 

The convergence rate (6.20) is the sharpest possible [10]. The rate (6.21) 

is not optimal. To obtain a faster rate of convergence for x and X, one can 

use -u. and v. in (DE) = 0 and (DE) . = 0 (1 S i S N) to solve for more 
h n i 


accurate x and X. 
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§7. Examples and Computational Results 

In this section, we apply the finite element method to some examples and 
present our numerical results. 

Example 1 We consider the following two person non zero-sum game 


f f • 


x(t) = x(t) + u 1 (t) + 2u 2 (t) + 1 , t € [0 ,t], T = ir/4, 

x(0) - 0 


J^Cx.u) = / [ j x( t ) + (cos t + ^)[ 2 + h | Ul (t)| 2 ] dt 


J (x,u) = Jf (|x(t) - sin t| 2 + 2 | u (t)| 2 ] dt. 




The Lagrangian L in (4.3) corresponding to this problem is 


(7.1) L(p 0 , P;L ,p 2 ) = " j <P Q +Pq» f(P 0 + Pq)> + f [< Pl + Pi>Pi + P 1 > + <P 2 + P 2 >P 2 +p 2 > ^ 


- \ <P Q +P X +P 2 » 4 (P Q +P X + P 2 )> + <P 0 + Pi + P 2 * 2 • P x +2 * p 2 > 


1 2 ' ’ 1 ~ — — — — 

- <p Q +p Q , j[(cos t+ j) +sin t]> - [<-p 1 +p 1 , cos t + - > + <p 2 +p 2 , s in t>] 


~ <Pq + Pi + P 2 > 1 > - \ < j[(cos t + j) + sin t ] , (cos t + y) + sin t> 


1 1 1 1 - 1 
+ yfccos t + — , cos t + y> + <sin t , sin t> ] . 

Uaing — = J , fl^ 1= l,-ffi 2 1 = 1, =■ 2, b 2 m 2 1b 2 = 2, S = 4^ we~ easily 

verify that (Al) - (A4) are all satisfied for all T > 0. 

We choose a (4,l)-system of Hermite cubic splines as in [l3 >p .56]. The interval 
[0,T] is divided into N equal subintervals, each with mesh length h = y. The matrix 
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is a (6N *3) x (6N f3) matrix. We use the IMSL high accuracy subroutine LE{J2S 

to solve the matrix equation \ on an IBM370/Model 3033 at the Pennsylvania 

State University. 

Numerical results are plotted in. Figures 1-4: 

(i) Figure 1: Strategy u^ is plotted, using h = 2/16, J/ 32, 2/ 64, respectively 

Numerical results for v^ are found to be identical with u^, as indicated 

in Theorem 4.7. 

(ii) Figure 2: Strategy u 2 is plotted, using h = 2/16, 2/32, 2/64, respectively 

Numerical results for v 2 are identical with u 2< 

(iii) Figure 3: State x is plotted, using h = 2/16, jr/32, 2/64. 

1 2 

(iv) Figure 4: x, x and x are plotted, with h = 2/16. Except near t = 0 

and t = T (where all three trajectories exhibit a great deal of roughness), 

1 2 

the numerical data of x, x and x differ very little. 

The values of L(pg,p^,p 2 ) and J(x,u;X,v) are found to be 

L = J = 0.02394619 , h = -?/16 

4 

(7.2) L = J = 0.01211985 , h = J/32 

L = J - 0.00609733 , h = -2/64. 

* 4 

A quick observation points out that L converges to 0 with rate 07(h ^ ) . This 
seems to contradict (6.14), which predicts that the rate should be h^). 
Nevertheless, we believe that this is not really paradoxical because, first 
of all, 0(h 6 ) is a quite high rate of convergence, which is hard to verify 
and, secondly, we believe that the values of L and J in (7.2) are probably 
composed of quadrature and round off errors, since our h is very small and 
the matrix solver has high accuracy. All of our calculations were carried 
out with double precision. 
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In Table 1, we list some values of x, x 1 , x 2 , p Q) 

and p 2 at certain selected nodal points. 

Example 2 We compute Example 1 again, but with T = 2 it and h = 2ir/16. 

The graphs for u^ and u^ are plotted in Figure 5. Hare again we have 

1 2 

V 1 = u l* v 2 = U '’ * n numer ^ ca ^ va lues. The graphs for x, x and x are 
plotted in Figure 6. The reader may compare them with the pictures of 
Example 1. 

Example 3 We consider the following 2-person non-zero sum game: 

x(t) = x(t) + cos t • u^(t) + sin t • u 2 (t) +1, 0 < t 5 T, 

x(0) = 0, 

T 

(x ,u ) h / [ |x(t) - d 1 (cos t + y) | 2 + J u^(t) ]dt, 

T 

J 2 (x,u)= / [ | x(t) - d 2 sin t(" + | u^tjjdt, 

It is not clear to us whether conditions (A2) - (A4) are satisfied when 
T is large. 

For (d^,d 2 ) - (-1,1) and T = ^ , we find that 

L = 0.02394619 , h = J/16 

L = 0.01211985 , h - |/32 

L = 0.00609733 , h = |/64. 

Surprisingly, they agree identically with the values in (7.2) (except 
the last few digits which have been rounded off by us). 

For 1 = 2 ^, Cd ^ j *^ 2 ^ = (“1» ^-9), we find that 


L = 

-0.02630621, 

h = 2TT/4 

L = 

-0.03772221, 

h » 2 tt/8 

L = 

-0.0412112/, 

h = 27T/16 

L = 

-0.04456356, 

h = 2TT/32 


L = 



-0.05005449, h = 2^/64 



61 


These values of L are all negative and seem to be divergent. See [3, S4^ 
Example 3] for further discussions. 


Due to the lack of any known closed form solutions to make comparisons, 
error estimates (6.20) and (6.21) can not be verified at this stage. However, 
in Part II [3] of our papers, numerical results for Example 1 will be compared 
with those obtained from another very different approach - the penalty method. 
They manifest remarkable agreement. This gives a good indication that our 
treatment and calculations are sound. 

Note added in proof: We have recently improved the order of convergence of 

L to which is close to the predicted rate h^) mentioned at the last 

paragraph of page 59. In addition, the roughness of the state x as well as x^ 
2 

and x as shown in Figures 3, 4, and 6, and those in certain figures in Part II 
of our papers, have all been eliminated. The improved numerical results will be 
published later on in a technical journal. 



(N 

\o 




II 

4-> 



‘-•H 

rr 

+ 


t = - • I 
4 4 

r 

H 

II 

t= lo- 
ll 

U 


h® J/16 

h- \m 

h- J/64 

h= J/16 

h= f/32 

h= J/64 

h- f/16 

h= J/32 


h= J/16 

h- 1/32 

h“ 1/6 l 1 
k 1 

U 1 

-2.022507 

-2.050318 


-1.199239 

-1.219113 

-1.229223 

-0.536199 

-0.549396 

-0.556116 

0.0 

0.0 

0.0 

U 2 

0.421594 

0.431190 

0.436094 

0.271250 

0.278159 

0.281693 

0.123312 

0.127565 

0.129746 

0.0 

0.0 

0.0 

X 

-0.131033 

-0.127947 


-0.138533 

-0.137644 

-0.137191 

-0.053930 

-0.053681 

-0.053709 

-0.25000 

-0.250000 


II 

-0.145114 

-0.134294 


-0.153802 

-0.145366 

-0.141075 

-0.073320 

-0.063079 

-0.058435 

-1.207107 

-1.207107 


x^ 

-0.116951 

-0.121600 


-0.123263 

-0.129922 

-0.133308 

-0.034540 

-0.044283 

-0.048983 

0.707107 

0.707107 


p o 

0.589659 

-0.593969 


-0.328370 

-0.331397 

-0.332918 

-0.144788 

-0.147134 

-0.148312 

0.0 

0.0 

0.0 

D 

1.011253 

1.025159 

1.032225 

0.599620 

0.609556 

0.614612 

0.268099 

0.274698 

0.278058 

0.0 

0.0 

0.0 

p 2 

-0.421594 

-0.431190 

-0.436094 

0.271250 

-0.278159 

-0.281694 

-0.123312 

-0.127565 

-0.129746 

0.0 

0.0 

0.0 


Remark: The numerical values of v^ , are identical, respectively, with u^, u^. 
off figures with six decimal place accuracy. 


All entries above are rounded 


Table 1 : 


Numerical Values of 


u 1 , u 2 , x, 


1 

x , 


X 2 


p Q , p 1 and p 2 


at t 


1 . jl JL , 2L 2 . JL 

4 4 ’ 2 4 ’ 4 4 



f 


> 
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Figure 3: Duality Solution of x. 

Example 1 



Throughout Figures 1, 2 and 3, curves 1, 2, and 3 represent the 
numerical solutions with h = £■/ 16 , |-/32 and respectively, 

for Example 1. 
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